In this paper, we address a rain removal problem from a single image, even inthe presence of heavy rain and rain streak accumulation. Our core ideas lie inthe new rain image models and a novel deep learning architecture. We firstmodify an existing model comprising a rain streak layer and a background layer,by adding a binary map that locates rain streak regions. Second, we create anew model consisting of a component representing rain streak accumulation(where individual streaks cannot be seen, and thus visually similar to mist orfog), and another component representing various shapes and directions ofoverlapping rain streaks, which usually happen in heavy rain. Based on thefirst model, we develop a multi-task deep learning architecture that learns thebinary rain streak map, the appearance of rain streaks, and the cleanbackground, which is our ultimate output. The additional binary map iscritically beneficial, since its loss function can provide additional stronginformation to the network. To handle rain streak accumulation (again, aphenomenon visually similar to mist or fog) and various shapes and directionsof overlapping rain streaks, we propose a recurrent rain detection and removalnetwork that removes rain streaks and clears up the rain accumulationiteratively and progressively. In each recurrence of our method, a newcontextualized dilated network is developed to exploit regional contextualinformation and outputs better representation for rain detection. Theevaluation on real images, particularly on heavy rain, shows the effectivenessof our novel models and architecture, outperforming the state-of-the-artmethods significantly. Our codes and data sets will be publicly available.
展开▼
